I am an associate professor at Carnegie Mellon University in the Machine Learning and the Computer Science departments. I work in the areas of machine learning, statistics, information theory and game theory. My current work addresses various biases and other challenges in human evaluations via principled and practical approaches. A focus application is scientific peer review, where our work has already made significant impact.

Survey on Systemic Challenges and Solutions on Bias and Unfairness in Peer Review, and associated tutorial slides

Blog on various aspects of academia, research, and peer review

Email: nihars [at] cs.cmu.edu
Office: GHC 8211




RESEARCH

My research focusses on two inter-related themes:

  1. Distributed human evaluations: Distributed human evaluations are integral to various applications where a set of items is assessed by a group of individuals, each person evaluating only a subset of the items and each item being evaluated by only a handful of individuals. This setup is commonly seen in domains such as scientific peer review, hiring and promotion processes, academic admissions, crowdsourcing initiatives, healthcare assessments, judicial verdicts, and online rating and recommendation systems, among others. The decentralization of evaluations, however, often leads to a host of challenges including instances of fraud, subjectivity, miscalibration, biases, breaches of privacy, and operational inefficiencies.
  2. Research on research Our research also focuses on problems in scientific research itself and its evaluations. The challenges within this domain hold substantial consequences. They impact the allocation of billions of dollars in annual grant awards, influence the trajectories of researchers' careers due to the "rich-gets-richer" phenomenon in academia, and can detrimentally affect the public's perception of science.
Our research addresses these important challenges at scale, in a principled and pragmatic manner. Our work is comprehensive, establishing fundamental limits, designing algorithms, deriving theoretical guaranees for them, conducting evaluations, and working with users for actual deployment and impact. The algorithms we have developed are now extensively used for peer review of tens of thousands of papers across computer science and various other fields. Additionally, our experiments have helped to shape the policies of numerous peer-review venues in an evidence-based manner. Finally, the impact of our work extends to domains beyond review of scientific papers.




PUBLICATIONS




GROUP
Justin Payan
Justin Payan
Postdoc, Machine Learning Department
Alexander Goldberg
Alexander Goldberg
PhD student, Computer Science Department
(advised jointly with Giulia Fanti)
Lucas (Yi) Li
Lucas (Yi) Li
MS student, Machine Learning
Sarina Xi
Sarina Xi
MS student, Machine Learning
Vishisht Rao
Vishisht Rao
MS student, Machine Learning
Orelia Pi
Orelia Pi
BS student, Computer Science

ALUMNI


Charvi Rastogi
Charvi Rastogi
PhD, Machine Learning Department
(advised jointly with Ken Holstein)
Steven Jecmen
Steven Jecmen
PhD, Computer Science Department
(advised jointly with Fei Fang)
Ivan Stelmakh
Ivan Stelmakh
PhD, Machine Learning Department
(advised jointly with Aarti Singh)
Jingyan Wang
Jingyan Wang
PhD, Robotics Institute
Janet Hsieh
Janet (Jhih-Yi) Hsieh
MS in Computer Science
(advised jointly with Aditi Raghunathan)
Ryan Liu
Ryan Liu
BS and MS in Computer Science
Carmel Baharav
Carmel Baharav
BS in Computer Science
Komal Dhull
Komal Dhull
BS in Computer Science
Wenxin Ding
Wenxin Ding
MS in Computer Science
BS in Mathematics and Computer Science
(advised jointly with Weina Wang)
Qiqi Xu
Qiqi Xu
MS in Machine Learning
(advised jointly with Hoda Heidari)


FUNDING
We gratefully acknowledge support from the National Science Foundation, CMU Block center, CMU CyLab, ONR, a Google Research Scholar award, a JP Morgan Faculty Research Award, and an NSF-Amazon Fair AI research grant.



TEACHING

Fall 2024 10-715 Advanced Introduction to Machine Learning
Spring 2024 15-281 Artificial Intelligence: Representation and Problem Solving
Fall 2023 10-715 Advanced Introduction to Machine Learning
Fall 2022 10-715 Advanced Introduction to Machine Learning
Spring 2022 15-780 Graduate Artificial Intelligence
Fall 2021 10-715 Advanced Introduction to Machine Learning
Spring 2021 15-780 Graduate Artificial Intelligence
Fall 2020 10-715 Advanced Introduction to Machine Learning
Spring 2020 15-780 Graduate Artificial Intelligence
Fall 2019 10-715 Advanced Introduction to Machine Learning
Spring 2019 15-780 Graduate Artificial Intelligence
Fall 2017 10-709 Fundamentals of Learning from the Crowd

In my spare time, I am also creating introductory machine learning short lectures in Hindi, accessible to anyone without requiring any math or programming knowledge: Link to Youtube videos



CURRICULUM VITAE
EDUCATION
HONORS

SERVICE (outside CMU)

INDUSTRY EXPERIENCE